Probability computation: The Language of Visual Perception?∗

We argue that the function of vision is to get correct and useful answers about the state of the world. However, given the state of the world is not uniquely specified by the visual input, the visual system must make inferences. Thus, theories of visual perception will be theories of inference, and we need a language in which theories of inference can be described. Analogous to calculus being a necessary language for physics, the language of Bayesian decision theory is necessary to describe how reliable answers about the world can be obtained from image patterns. A Bayesian language is not by itself a testable theory of vision, but can be used to formulate testable theories. The test for a language is utility and completeness for deriving predictive theories. We argue the Bayesian language provides necessary formalism to deal with the sophistication and flexibility of perception that has been missing from some other approaches. Key missing elements include the ability to model uncertainty, an emphasis on the probabilistic modeling of pattern synthesis as a necessary prerequisite to understanding pattern inference, and the propagation of the probability distributions of scene properties, rather than just estimates. We show that this approach, called Pattern theory, is a radical generalization of the communication theory component of classical signal detection theory. This generalization unifies signal detection theory, ideal observer analysis, and inferential approaches to perception, thus overcoming the severe limitations in information modeling suffered by classical SDT. We distinguish features of the Bayesian approach, disabuse misconceptions about Bayesian theories of vision, and show how many areas of perceptual study fit into the Bayesian framework. 1 Perception is pattern decoding Few would dispute the view that visual perception is the brain’s process for arriving at useful information about the world from images. Divergent opinions, however, have been expressed over how to describe the computations (or lack thereof) for functional visual behavior. Visual perception has been described as unconscious inference [41, 36], reconstruction[18], resonance [32], problem solving [78], computation [64], and more recently as Bayesian inference [52]. In part, the debate gets muddled due to lack of a wellspecified explanatory goal and level of abstraction. To clarify, we see the grand challenge to be theories of visual performance given the complexities of natural images and the richness of visual behavior. But here the level of explanation is crucial: if our theories are too abstract, we lose the specificity of quantitative predictions; if the theories are too fine-grained, the model mechanisms for natural pattern processing will be too complex to test. Our proposed strategy follows that of statistical mechanics. Few physicists doubt that the large-scale properties of physical systems rest on the lawful function of individual molecules, just as few brain scientists doubt that an organism’s behavior depends on the lawful function of neurons. Physicists would agree that the modeling level has to be appropriate to the measurements and phenomena of large-scale systems; thus statistical mechanics links molecular kinetics to thermodynamics. Although the bridge between neurons and system behavior has yet to be built, the language of Bayesian statistics provides the level of description analogous to thermodynamics 1. For vision, theories at this level are testable at the level of constraints, and are less committal about representations, algorithms, or mechanisms. The purpose of this chapter is to specify the fundamental principles required to face the grand chalOur level of analysis falls between the computational/function and representation/algorithmic levels in the Marr hierarchy